1. Technical Field
The present disclosure generally relates to virtual execution environments, and more particularly to virtual execution environments for streaming languages.
2. Discussion of Related Art
In data stream processing, data flows through a data flow graph, where each directed edge is a FIFO (first-in-first-out) queue, and each vertex is an operator that processes streaming data on-the-fly. This computing model is characterized by data in motion, since data continuously flows on queues between operators, and typically the total volume of data over time far exceeds the storage capacity of any single machine. Each operator repeatedly fires to consume a small amount of data from its input queues(s) and produce data on its output queue(s). The operators in the graph typically run in parallel with each other, making data stream processing amenable for high performance computing tasks on parallel or distributed hardware. Programs typically run for weeks or months, though we will also consider the special case of batch jobs that only run on the order of a few minutes, if those batch jobs fit the streaming model.
A streaming language is a programming language designed to express streaming applications. Streaming languages have features that directly represent the data flow graph, where the primary means of data exchange between operators is through FIFO queues. Example streaming languages include but are not limited to StreamIt, CQL, and Sawzall.
The StreamIt language has been used for applications such as video streaming. A tape in StreamIt terminology is a queue, and a filter is an operator. The user implements filters and arranges them in a data flow graph. Each time a filter fires, it pops some data from its input tape, does some computation, then pushes some data on its output tape.
CQL, the Continuous Query Language, has been used for applications such as road traffic monitoring. CQL is a dialect of the widely-used database language SQL, and as such, CQL includes a set of relational operators.
The Sawzall language has been used for batch applications such as web content analysis. Even though it is intended for batch jobs, it exhibits streaming characteristics, since data flows over a data flow graph of queues through operators. In Sawzall, the user implements a map operator and selects an aggregator operator from a built-in library.
According to an embodiment of the present disclosure, a need exists for a method for implementing multiple streaming languages, with multiple optimizations, running on multiple platforms.